Understanding Individual Subject Differences through Large Behavioral Datasets: Analytical and Statistical Considerations.

A core feature of behavior analysis is the single-subject design, in which each subject serves as its own control. This approach is powerful for identifying manipulations that are causal to behavioral changes but often fails to account for individual differences, particularly when coupled with a small sample size. It is more common for other subfields of psychology to use larger-N approaches; however, these designs also often fail to account for the individual by focusing on aggregate-level data only. Moving forward, it is important to study individual differences to identify subgroups of the population that may respond differently to interventions and to improve the generalizability and reproducibility of behavioral science. We propose that large-N datasets should be used in behavior analysis to better understand individual subject variability. First, we describe how individual differences have been historically treated and then outline practical reasons to study individual subject variability. Then, we describe various methods for analyzing large-N datasets while accounting for the individual, including correlational analyses, machine learning, mixed-effects models, clustering, and simulation. We provide relevant examples of these techniques from published behavioral literature and from a publicly available dataset compiled from five different rat experiments, which illustrates both group-level effects and heterogeneity across individual subjects. We encourage other behavior analysts to make use of the substantial advancements in online data sharing to compile large-N datasets and use statistical approaches to explore individual differences.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies.

BACKGROUND: The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status. METHODS: We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results. RESULTS: The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]). CONCLUSIONS: Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed. TRIAL REGISTRATION: https://osf.io/q5hcs.

A survey of experimental stimulus presentation code sharing in major areas of psychology.

Computer code plays a vital role in modern science, from the conception and design of experiments through to final data analyses. Open sharing of code has been widely discussed as being advantageous to the scientific process, allowing experiments to be more easily replicated, helping with error detection, and reducing wasted effort and resources. In the case of psychology, the code used to present stimuli is a fundamental component of many experiments. It is not known, however, the degree to which researchers are sharing this type of code. To estimate this, we conducted a survey of 400 psychology papers published between 2016 and 2021, identifying those working with the open-source tools Psychtoolbox and PsychoPy that openly share stimulus presentation code. For those that did, we established if it would run following download and also appraised the code's usability in terms of style and documentation. It was found that only 8.4% of papers shared stimulus code, compared to 17.9% sharing analysis code and 31.7% sharing data. Of shared code, 70% ran directly or after minor corrections. For code that did not run, the main error was missing dependencies (66.7%). The usability of the code was moderate, with low levels of code annotation and minimal documentation provided. These results suggest that stimulus presentation code sharing lags behind other forms of code and data sharing, potentially due to less emphasis on such code in open-science discussions and in journal policies. The results also highlight a need for improved documentation to maximize code utility.

Gradients of Brain Organization: Smooth Sailing from Methods Development to User Community.

Multimodal neuroimaging grants a powerful in vivo window into the structure and function of the human brain. Recent methodological and conceptual advances have enabled investigations of the interplay between large-scale spatial trends - or gradients - in brain structure and function, offering a framework to unify principles of brain organization across multiple scales. Strong community enthusiasm for these techniques has been instrumental in their widespread adoption and implementation to answer key questions in neuroscience. Following a brief review of current literature on this framework, this perspective paper will highlight how pragmatic steps aiming to make gradient methods more accessible to the community propelled these techniques to the forefront of neuroscientific inquiry. More specifically, we will emphasize how interest for gradient methods was catalyzed by data sharing, open-source software development, as well as the organization of dedicated workshops led by a diverse team of early career researchers. To this end, we argue that the growing excitement for brain gradients is the result of coordinated and consistent efforts to build an inclusive community and can serve as a case in point for future innovations and conceptual advances in neuroinformatics. We close this perspective paper by discussing challenges for the continuous refinement of neuroscientific theory, methodological innovation, and real-world translation to maintain our collective progress towards integrated models of brain organization.

A pathway to strengthening open science: comments on the draft South African Ethics in Health Research Guidelines.

The recently released draft South African Ethics in Health Research Guidelines: Principles, Processes and Structures (Draft Guidelines) by the National Health Research Ethics Council recognize open data and provide guiding principles for this in the context of health research in South Africa. While its inclusion is a positive development, there is room for improvement. Although the Draft Guidelines leverage the Draft National Policy on Data and Cloud, it lacks incorporation of other relevant government policies, notably the Draft National Open Science Policy, and fails to sufficiently detail the principles of open science and open access. This limited scope and lack of comprehensive definition and detailed guidance present challenges for researchers in conducting ethical and responsible health research in South Africa. It constrains the Draft Guidelines from fully aligning with national imperatives and from fostering African-centric approaches. To address these issues, it is recommended that the Draft Guidelines integrate broader policies and principles, enhance clarity through comprehensive definitions, provide detailed guidance on open access, and promote African-centric approaches. Implementing these solutions will strengthen the Draft Guidelines, aligning them with national visions of open science, and thereby harnessing the full potential of South Africa's diverse scientific community in advancing health research.

State of open science in cancer research.

PURPOSE: This study has been focused on assessing the Open Science scenario of cancer research during the period 2011-2021, in terms of the derived scientific publications and raw data dissemination. METHODS: A cancer search equation was executed in the Science Citation Index-Expanded, collecting the papers signed by at least one Spanish institution. The same search strategy was performed in the Data Citation Index to describe dataset diffusion. RESULTS: 50,822 papers were recovered, 71% of which belong to first and second quartile journals. 59% of the articles were published in Open Access (OA) journals. The Open Access model and international collaboration positively conditioned the number of citations received. Among the most productive journals stood out Plos One, Cancers, and Clinical and Translational Oncology. 2693 genomics, proteomics and metabolomics datasets were retrieved, being Gene Expression Omnibus the favoured repository. CONCLUSIONS: There has been an increase in oncology publications in Open Access. Most were published in first quartile journals and received higher citations than non-Open Access articles, as well as when oncological investigation was performed between international research teams, being relevant in the context of Open Science. Genetic repositories have been the preferred for sharing oncology datasets. Further investigation of research and data sharing in oncology is needed, supported by stronger Open Science policies, to achieve better data sharing practices among three scientific main pillars: researchers, publishers, and scientific organizations.

Biodiversity dataset and atlas of the special area of conservation Montesinho/Nogueira, Portugal.

Background: The special area of conservation Montesinho/Nogueira (SAC-MN) is a key area for biodiversity conservation in the Iberian Peninsula. Covering an area of approximately 1081 km² in the northeast of Portugal mainland, the SAC-MN is home to a wide range of species, including several endemic and endangered species and priority habitats. Despite its ecological significance and importance for conservation, there is a lack of publicly available biodiversity data, which urges the need to create a comprehensive and up-to-date biodiversity dataset for the SAC-MN. New information: To bridge the knowledge gap on biodiversity in SAC-MN, we undertook a thorough data collection process, including species occurrence records and conservation status information at regional (Portugal) and European levels, from multiple sources. We collected and compiled this information for five major taxonomic groups (amphibians, birds, flora - vascular plants, mammals and reptiles) in SAC-MN, resulting in a total of 31,871 records with 1,312 documented species. In addition, we developed an easy-to-navigate web-based geographic information system (Web GIS). In this article, we present an in-depth report on the process of compiling and preparing data, as well as the development and design of our Web GIS to increase awareness and enhance understanding of the importance of preserving biodiversity in SAC-MN.

A practical guide to cross-cultural and multi-sited data collection in the biological and behavioural sciences.

Researchers in the biological and behavioural sciences are increasingly conducting collaborative, multi-sited projects to address how phenomena vary across ecologies. These types of projects, however, pose additional workflow challenges beyond those typically encountered in single-sited projects. Through specific attention to cross-cultural research projects, we highlight four key aspects of multi-sited projects that must be considered during the design phase to ensure success: (1) project and team management; (2) protocol and instrument development; (3) data management and documentation; and (4) equitable and collaborative practices. Our recommendations are supported by examples from our experiences collaborating on the Evolutionary Demography of Religion project, a mixed-methods project collecting data across five countries in collaboration with research partners in each host country. To existing discourse, we contribute new recommendations around team and project management, introduce practical recommendations for exploring the validity of instruments through qualitative techniques during piloting, highlight the importance of good documentation at all steps of the project, and demonstrate how data management workflows can be strengthened through open science practices. While this project was rooted in cross-cultural human behavioural ecology and evolutionary anthropology, lessons learned from this project are applicable to multi-sited research across the biological and behavioural sciences.

Adapting Open Science and Pre-registration to Longitudinal Research.

Open science practices, such as pre-registration and data sharing, increase transparency and may improve the replicability of developmental science. However, developmental science has lagged behind other fields in implementing open science practices. This lag may arise from unique challenges and considerations of longitudinal research. In this paper, preliminary guidelines are provided for adapting open science practices to longitudinal research to facilitate researchers' use of these practices. The guidelines propose a serial and modular approach to registration that includes an initial pre-registration of the methods and focal hypotheses of the longitudinal study, along with subsequent pre- or co-registered questions, hypotheses, and analysis plans associated with specific papers. Researchers are encouraged to share their research materials and relevant data with associated papers, and to report sufficient information for replicability. In addition, there should be careful consideration about requirements regarding the timing of data sharing, to avoid disincentivizing longitudinal research.

Design and validation of a conceptual model regarding impact of open science on healthcare research processes.

INTRODUCTION: The development and use of digital tools in various stages of research highlight the importance of novel open science methods for an integrated and accessible research system. The objective of this study was to design and validate a conceptual model of open science on healthcare research processes. METHODS: This research was conducted in three phases using a mixed-methods approach. The first phase employed a qualitative method, namely purposive sampling and semi-structured interview guides to collect data from healthcare researchers and managers. Influential factors of open science on research processes were extracted for refining the components and developing the proposed model; the second phase utilized a panel of experts and collective agreement through purposive sampling. The final phase involved purposive sampling and Delphi technique to validate the components of the proposed model according to researchers' perspectives. FINDINGS: From the thematic analysis of 20 interview on the study topic, 385 codes, 38 sub-themes, and 14 main themes were extracted for the initial proposed model. These components were reviewed by expert panel members, resulting in 31 sub-themes, 13 main themes, and 4 approved themes. Ultimately, the agreed-upon model was assessed in four layers for validation by the expert panel, and all the components achieved a score of > 75% in two Delphi rounds. The validated model was presented based on the infrastructure and culture layers, as well as supervision, assessment, publication, and sharing. CONCLUSION: To effectively implement these methods in the research process, it is essential to create cultural and infrastructural backgrounds and predefined requirements for preventing potential abuses and privacy concerns in the healthcare system. Applying these principles will lead to greater access to outputs, increasing the credibility of research results and the utilization of collective intelligence in solving healthcare system issues.

Improving the integrity and reproducibility of research that uses antibodies: a technical, data sharing, behavioral and policy challenge.

Antibodies are one of the most important reagents used in biomedical and fundamental research, used to identify, and quantify proteins, contribute to knowledge of disease mechanisms, and validate drug targets. Yet many antibodies used in research do not recognize their intended target, or recognize additional molecules, compromising the integrity of research findings and leading to waste of resources, lack of reproducibility, failure of research projects, and delays in drug development. Researchers frequently use antibodies without confirming that they perform as intended in their application of interest. Here we argue that the determinants of end-user antibody choice and use are critical, and under-addressed, behavioral drivers of this problem. This interacts with the batch-to-batch variability of these biological reagents, and the paucity of available characterization data for most antibodies, making it more difficult for researchers to choose high quality reagents and perform necessary validation experiments. The open-science company YCharOS works with major antibody manufacturers and knockout cell line producers to characterize antibodies, identifying high-performing renewable antibodies for many targets in neuroscience. This shows the progress that can be made by stakeholders working together. However, their work so far applies to only a tiny fraction of available antibodies. Where characterization data exists, end-users need help to find and use it appropriately. While progress has been made in the context of technical solutions and antibody characterization, we argue that initiatives to make best practice behaviors by researchers more feasible, easy, and rewarding are needed. Global cooperation and coordination between multiple partners and stakeholders will be crucial to address the technical, policy, behavioral, and open data sharing challenges. We offer potential solutions by describing our Only Good Antibodies initiative, a community of researchers and partner organizations working toward the necessary change. We conclude with an open invitation for stakeholders, including researchers, to join our cause.

CREDO: a friendly Customizable, REproducible, DOcker file generator for bioinformatics applications.

BACKGROUND: The analysis of large and complex biological datasets in bioinformatics poses a significant challenge to achieving reproducible research outcomes due to inconsistencies and the lack of standardization in the analysis process. These issues can lead to discrepancies in results, undermining the credibility and impact of bioinformatics research and creating mistrust in the scientific process. To address these challenges, open science practices such as sharing data, code, and methods have been encouraged. RESULTS: CREDO, a Customizable, REproducible, DOcker file generator for bioinformatics applications, has been developed as a tool to moderate reproducibility issues by building and distributing docker containers with embedded bioinformatics tools. CREDO simplifies the process of generating Docker images, facilitating reproducibility and efficient research in bioinformatics. The crucial step in generating a Docker image is creating the Dockerfile, which requires incorporating heterogeneous packages and environments such as Bioconductor and Conda. CREDO stores all required package information and dependencies in a Github-compatible format to enhance Docker image reproducibility, allowing easy image creation from scratch. The user-friendly GUI and CREDO's ability to generate modular Docker images make it an ideal tool for life scientists to efficiently create Docker images. Overall, CREDO is a valuable tool for addressing reproducibility issues in bioinformatics research and promoting open science practices.

An ode to multidisciplinarity: the 'Bacteria Orchestrate Life' international meeting.

If scientists stick to their research expertise without collaborating with other experts in different fields, it could stall the progress of their work in a world where interdisciplinary thinking and working should be second nature. Biologists are at the forefront of this trend. That is why a consortium formed by the faculty of sciences of Tunis El Manar University, Tunisia, the GetGenome Foundation and Learn and Win, decided to organise an international conference on bacteria, a perfect field for multidisciplinarity. For 3 days, from 14 to 16 December 2023, more than 200 interdisciplinary researchers and students of life sciences and more than 20 international speakers and trainers met at the faculty of sciences in Tunis, to discuss microbiology and bacteria from different horizons, from the most fundamental to the most imaginative, with a strong focus on technologies and discoveries. This Meeting Review describes the scientific event and highlights the main results of both the conferences and the practical sessions.

The power of many brains: Catalyzing neuropsychiatric discovery through open neuroimaging data and large-scale collaboration.

Recent advances in open neuroimaging data are enhancing our comprehension of neuropsychiatric disorders. By pooling images from various cohorts, statistical power has increased, enabling the detection of subtle abnormalities and robust associations, and fostering new research methods. Global collaborations in imaging have furthered our knowledge of the neurobiological foundations of brain disorders and aided in imaging-based prediction for more targeted treatment. Large-scale magnetic resonance imaging initiatives are driving innovation in analytics and supporting generalizable psychiatric studies. We also emphasize the significant role of big data in understanding neural mechanisms and in the early identification and precise treatment of neuropsychiatric disorders. However, challenges such as data harmonization across different sites, privacy protection, and effective data sharing must be addressed. With proper governance and open science practices, we conclude with a projection of how large-scale imaging resources and collaborations could revolutionize diagnosis, treatment selection, and outcome prediction, contributing to optimal brain health.

A registered report survey of open research practices in psychology departments in the UK and Ireland.

Open research practices seek to enhance the transparency and reproducibility of research. While there is evidence of increased uptake in these practices, such as study preregistration and open data, facilitated by new infrastructure and policies, little research has assessed general uptake of such practices across psychology university researchers. The current study estimates psychologists' level of engagement in open research practices across universities in the United Kingdom and Ireland, while also assessing possible explanatory factors that may impact their engagement. Data were collected from 602 psychology researchers in the United Kingdom and Ireland on the extent to which they have implemented various practices (e.g., use of preprints, preregistration, open data, open materials). Here we present the summarized descriptive results, as well as considering differences between various categories of researcher (e.g., career stage, subdiscipline, methodology), and examining the relationship between researcher's practices and their self-reported capability, opportunity, and motivation (COM-B) to engage in open research practices. Results show that while there is considerable variability in engagement of open research practices, differences across career stage and subdiscipline of psychology are small by comparison. We observed consistent differences according to respondent's research methodology and based on the presence of institutional support for open research. COM-B dimensions were collectively significant predictors of engagement in open research, with automatic motivation emerging as a consistently strong predictor. We discuss these findings, outline some of the challenges experienced in this study, and offer suggestions and recommendations for future research. Estimating the prevalence of responsible research practices is important to assess sustained behaviour change in research reform, tailor educational training initiatives, and to understand potential factors that might impact engagement.

The challenges of open data sharing for qualitative researchers.

'Open Science' advocates for open access to scientific research, as well as sharing data, analysis plans and code in order to enable replication of results. However, these requirements typically fail to account for methodological differences between quantitative and qualitative research, and serious ethical problems are raised by the suggestion that full qualitative datasets can or should be published alongside qualitative research papers. Aside from important ethical concerns, the idea of sharing qualitative data in order to enable replication is conceptually at odds with the underpinnings on most qualitative methodologies, which highlight the importance of the unique interpretative function of the researcher. The question of whether secondary analysis of qualitative data is acceptable is key, and in this commentary we argue that there are good conceptual, ethical and economic reasons to consider how funders, researchers and publishers can make better use of existing data.

A dataset without a code book: ethnography and open science.

This paper reflects upon calls for "open data" in ethnography, drawing on our experiences doing research on sexual violence. The core claim of this paper is not that open data is undesirable; it is that there is a lot we must know before we presume its benefits apply to ethnographic research. The epistemic and ontological foundation of open data is grounded in a logic that is not always consistent with that of ethnographic practice. We begin by identifying three logics of open data-epistemic, political-economic, and regulatory-which each address a perceived problem with knowledge production and point to open science as the solution. We then evaluate these logics in the context of the practice of ethnographic research. Claims that open data would improve data quality are, in our assessment, potentially reversed: in our own ethnographic work, open data practices would likely have compromised our data quality. And protecting subject identities would have meant creating accessible data that would not allow for replication. For ethnographic work, open data would be like having the data set without the codebook. Before we adopt open data to improve the quality of science, we need to answer a series of questions about what open data does to data quality. Rather than blindly make a normative commitment to a principle, we need empirical work on the impact of such practices - work which must be done with respect to the different epistemic cultures' modes of inquiry. Ethnographers, as well as the institutions that fund and regulate ethnographic research, should only embrace open data after the subject has been researched and evaluated within our own epistemic community.

DateLife: leveraging databases and analytical tools to reveal the dated Tree of Life.

Chronograms -phylogenies with branch lengths proportional to time- represent key data on timing of evolutionary events for the study of natural processes in many areas of biological research. Chronograms also provide valuable information that can be used for education, science communication, and conservation policy decisions. Yet, achieving a high-quality reconstruction of a chronogram is a difficult and resource-consuming task. Here we present DateLife, a phylogenetic software implemented as an R package and an R Shiny web application available at www .datelife.org, that provides services for efficient and easy discovery, summary, reuse, and reanalysis of node age data mined from a curated database of expert, peer-reviewed, and openly available chronograms. The main DateLife workflow starts with one or more scientific taxon names provided by a user. Names are processed and standardized to a unified taxonomy, allowing DateLife to run a name match across its local chronogram database that is curated from Open Tree of Life's phylogenetic repository, and extract all chronograms that contain at least two queried taxon names, along with their metadata. Finally, node ages from matching chronograms are mapped using the congruification algorithm to corresponding nodes on a tree topology, either extracted from Open Tree of Life's synthetic phylogeny or one provided by the user. Congruified node ages are used as sec- ondary calibrations to date the chosen topology, with or without initial branch lengths, using different phylogenetic dating methods such as BLADJ, treePL, PATHd8 and MrBayes. We performed a cross-validation test to compare node ages resulting from a DateLife analysis (i.e, phylogenetic dating using secondary calibrations) to those from the original chronograms (i.e, obtained with primary calibrations), and found that DateLife's node age estimates are consistent with the age estimates from the original chronograms, with the largest variation in ages occurring around topologically deeper nodes. Because the results from any software for scientific analysis can only be as good as the data used as input, we highlight the importance of considering the results of a DateLife analysis in the context of the input chronograms. DateLife can help to increase awareness of the existing disparities among alternative hypotheses of dates for the same diversification events, and to support exploration of the effect of alternative chronogram hypotheses on downstream analyses, providing a framework for a more informed interpretation of evolutionary results.